This invention relates generally to multi-user virtual reality systems. More particularly, the invention relates to a multi-player gaming system for use over the Internet by numerous remotely connected users.
The Internet has become a popular medium through which much of our traditional social functions are being conducted. E-commerce applications are making personal shopping business-to-business transactions and interpersonal communication easier than ever. Internet-based electronic auctions allow professionals and individuals to post items for sale onto an electronic auction block for which other members of the Internet community may provide competitive bid prices. Electronic interpersonal communications have become common place as individuals and corporations communicate and conduct business with one another through e-mail, online telephony, video conferencing, and other new emerging communication products employing the Internet.
Despite the widespread acceptance of the Internet, the majority of Internet communications constitutes point-to-point communications that do not occur in real-time. Such point-to-point communication occurs when a single entity (person or business) communicates with only one other entity. Thus, electronic point-to-point conversations do not occur in real time and are not available to be seen or heard by anyone other than the two participants within a particular communications domain.
In an electronic auction context, a single server computer may be used to list a particular item for worldwide bidding. However, the multiple users of the electronic auction system do not interact with one another simultaneously and in real-time, as would typically be the case when an item is introduced on an auction block in the real world. Simultaneous, real-time visual and aural perception of large multi-user virtual communities have heretofore not been provided for by any software or computer systems currently in use on the Internet.
Avatar-based chat rooms and shopping malls are examples of Internet-based multi-user systems in which relatively small numbers of simultaneous users communicate with each other over the Internet. For example, ACTIVEWORLDS.COM provides a realistic, 3-D virtual reality chat service in which virtual reality worlds are created and avatar-based caricatures correspond with each other in a 3-D graphical environment. An xe2x80x9cavatarxe2x80x9d, as used herein, refers to the physical incarnation of an online user in the virtual reality world. The avatar may be a scanned image of the user""s face, for example, or a more complicated computer-generated charicature for use by the virtual reality participant. Such systems are limited, however, in that only a relatively small number of simultaneous participants typically communicate at any one time.
Further, a practical graphical limit to the number of simultaneous virtual reality users is present with respect to various aspects of the transactional ability of virtual reality computer systems. One difficulty is that a large a number of users will typically overrun the ability of any system to provide simultaneous, real-time communication and interaction particularly when graphics and three dimensional (xe2x80x9c3Dxe2x80x9d) avatars and environments are involved. Further complicating these limitations are the computational problems related to the number of multi-point users who may need to perceive one another. These problems increase exponentially with the number of participants. Therefore, a need exists for computer software and hardware systems directed to a large scale multi-user transaction system that facilitates online communication between multiple parties on a simultaneous, real-time basis. A large scale multi-user system of the type needed would support online user communities in which numerous simultaneous users are present within the community and are capable of both aural and visual perception.
One complication in the implementation of a massively multi-player interactive game is the design and implementation of a computer system which can efficiently administer thousands of remote participants in an online virtual community. Two problems to be solved in designing such a system include: (1) creating an efficient system architecture for supporting a large number of simultaneous users; and (2) load balancing the users"" transactions among computer servers. Typical computer systems will load balance the number of transactions evenly across all computer servers. This load balancing arrangement may not be desirable in a computer system implementing a virtual reality environment, however, since each server would have to possess a replication of the entire virtual reality in all its transactional variation. Thus, a need exists for a system that can efficiently implement and manage a massive multi-player game of the type described.
Other problems encountered in the design of such a multi-player virtual reality system are the difficulties in designing and implementing the software and computer systems used to monitor the virtual reality community. Much like the real world, a structure for administering to problems and rogue conduct of the users within virtual reality needs to be provided for the numerous users. No centralized monitoring system is currently available in a large scale system to simultaneously answer users"" questions, monitor and respond to users"" actions and communicate with users of the virtual community on a real-time basis. As a practical matter, administering and monitoring the virtual reality world of thousands of users, including a problem resolution system, system integrity issues, transmission bottleneck detection and elimination and other aspects of the system administration, need to be carefully regulated so that the system runs efficiently and the virtual reality experience provided to the numerous users is as realistic as possible.
Another barrier to the overall usability of a large scale networked multi-user virtual reality system is the relatively large amount of data that the system needs to present to each user. The data is necessary to accommodate the flow of information occurring between participants and the system servers and provide a good graphical rendering of that xe2x80x9cpersonxe2x80x9d or avatar. A massive multi-user environment that can accommodate thousands of users must overcome problems associated with providing such information to all users through the administrative computer system so that a virtual reality environment is created. Thus, a need exists for a virtual reality computing system that minimizes the amount of data being transmitted to each of the many users so as to make efficient use of both the computer system""s resources and the telecommunications bandwidth that connects the users to the administrative computer system.
In an online virtual reality environment, the transmitted data may include voice or test data for the purpose of permitting the users within the game to communicate with each other and with the system itself. Where the virtual reality environment is a game, such as a role playing game, the data may also include movement commands issued by the players to advance their avatar within the virtual reality environment. Due to the large number of players connected to the game at any point in time, it is advantageous to minimize the size and the latency of the data transmission between the players and the game servers while at the same time preserving some capability to attempt retransmissions for failed data transmissions.
Existing connection oriented protocols, such as the transmission control protocol (xe2x80x9cTCPxe2x80x9d) operating over the internet protocol (xe2x80x9cIPxe2x80x9dxe2x80x94or xe2x80x9cTCP/IPxe2x80x9d) provide for robust error recovery capabilities. In particular, application messages sent as TCP/IP packets include a sequence number as one element of the TCP/IP packet header, or protocol data unit (xe2x80x9cPDUxe2x80x9d). The entire message is then sent in sequence as a series of TCP/IP datagrams or segments containing both the TCP/IP PDU and the actual data. An acknowledge message is returned to the sending computer by the receiving computer for each message for which the entire series of segments have been received. If any of the segments are lost in the transmission process, the receiving computer transmits a signal requesting the missing segment. The sending computer responds by retransmitting only the missing segment. As a consequence of this reliability, TCP is inefficient in that the entire sequence of segments must be received by the receiving machine before the message is assembled at the receiving end, even if the message arrives out of turn with other messages. The later assembly of the message may render it outdated or irrelevant in view of later sent, but earlier received messages. TCP/IP does not provide for a independent control over the retransmission capabilities of the message.
As an alternative transport mechanism to TCP, a protocol known as the User Datagram Protocol (xe2x80x9cUDPxe2x80x9d) may be used between computers. A drawback of UDP is that it does not include any error recovery capability for lost datagrams. Instead, UDP sends outgoing datagrams without regard for actual receipt of those datagrams by the receiving machine. While UDP is simple to implement and low in transmission overhead, it does not provide enough error recovery to be useful in an interactive gaming environment that requires some level of certainty that transmitted game information has been received.
Thus, a need also exists for an alternative data transmission protocol to provide a selective and controllable acknowledgement function so as to reliably transmit information between the players and the game servers while at the same time maintaining control of the retransmission of the information.
A player""s enjoyment in participating in an online multi-player game is directly related to the quality of the game playing experience, which depends on various factors such as the graphics, audio and interactive activities provided by the game application software. The quality of the graphical presentation, in turn, depends in part on the game software and in part on the quality of the network connection linking the player""s PC and the game computer server.
In present online game systems, when a problem arises in the quality of graphics or movement of an avatar, there is no way for the player to know whether such problem is due to a software bug or a network connection. Therefore, a need exists for a graphical indicator to provide the game player with a graphical indication of the quality of the network connection linking the player to the game server so that the player will know if a problem is caused by the game software or the network connection.
Problems related to the rendering of the avatars within the virtual reality also exist due to the limited transmission bandwidth between each user and the administering computer system. In particular, varying levels of detail (xe2x80x9cLODsxe2x80x9d) are needed to efficiently render a realistic representation in a system where there are many participants in a virtual reality environment. The detail of the image is based on the users"" proximity with respect to one another and with respect to inanimate objects presented by the system computers in rendering the virtual reality world. In addition, reliable messaging transport mechanisms must be used between a user and the administrative system so that movements of the characters through the virtual reality world are easily and quickly transmitted to the server without incurring large latencies or delay times.
The details further from the player""s avatar do not need to be refreshed as frequently as those nearer the virtual character. Likewise, details outside the view of the player""s avatar also may not need to be refreshed as frequently as those within the avatar""s virtual view. An avatar, like a real person, however, is constantly moving within its environment. This poses problems with regard to updating the field of view with respect to moving playersxe2x80x94a problem that is compounded by the potentially large number of players moving in a multi-player environment. Therefore, a need exists for an improved system for providing and updating details in an environment in which a virtual character""s view is constantly changing while limiting the transmission bandwidth between the players"" computers and the game computer server while maintaining a realistic game appearance for the players.
For these and other reasons, there is a need for a massive multi-user environment administered by the computer system of the present invention so as to facilitate interpersonal interactions in a virtual reality world.
The system and methods of the present invention are intended to be generally applicable to any virtual reality system capable of supporting numerous remote users. In a preferred embodiment, the system is capable of simultaneously supporting thousands of users. Some features of the preferred embodiments described herein may relate to a multi-player online gaming system sold under the EverQuest(trademark) trademark by Sony Online Entertainment, a division of Sony computer Entertainment America Inc.
Those of ordinary skill in the art will understand that the description below provides exemplary embodiments of the invention and does not limit the scope of the invention to the specific embodiments herein. In particular, various features of the large scale multi-user system and methods described herein may be applied to other online virtual reality environments such as chat rooms, shopping malls and many other virtual reality scenarios in which numerous simultaneous users need to perceive and/or communicate with each other in real-time.
The terms xe2x80x9cuserxe2x80x9d and xe2x80x9cplayerxe2x80x9d are used interchangeably herein to refer to the actual person who participates in the virtual reality environment. Similarly, the terms avatar and character are used interchangeably to refer to the representation of the user within the virtual reality environment. Each computer or computer server may be a multipurpose or special purpose computer of varying size and transactional capacity depending on the task they are to perform. As used herein, the terms xe2x80x9ccomputerxe2x80x9d and xe2x80x9cuser computerxe2x80x9d are intended to cover any multi-purpose computer, hand held device, game console or any other device that permits a user to connect to and communicate with the multi-user virtual reality game of the present invention. Each computer used to connect a player to the virtual reality environment may be considered a xe2x80x9cclient.xe2x80x9d The term xe2x80x9cworldxe2x80x9d should be construed broadly herein to include various locations and environments within the virtual reality environment (e.g., geographical areas that may or may not overlap).
In a preferred embodiment of the invention, a computer system is provided for implementing a virtual reality environment, programmed in software and supporting a multi-user community. The computer system preferably includes a computer server complex including at least one central world server and a plurality of area servers, all of the servers running software to provide the virtual reality environment. The central world server including a processor and running software responsible for administering a unique virtual reality world within the virtual reality environment. The area servers including a processor and running software responsible for administering virtual geographic areas within the at least one virtual world of the virtual reality environment.
It is also preferable for the present computer system to include a plurality of computer server complexes, each of the computer server complexes running the software so as to provide a plurality of the virtual reality worlds.
The present computer system may also include a local area network for connecting a plurality of computer server complexes. A telecommunications complex is preferably connected to the local area network for providing a connection to a telecommunications network including a plurality of user computers (i.e., clients), each having a processor for running software that provides an interface to the virtual reality environment. The plurality of computers are used by players within a network community for interacting in the virtual reality environment.
Another aspect of the invention provides a method of administering a computer system for implementing a virtual reality environment and associated computer readable media, the virtual reality environment being programmed in software and supporting a multi-user community. The computer system preferably includes a login server, at least one world server and plurality of area servers that manipulate avatars. Each of the world servers preferably include a processor for running software responsible for administering a unique virtual world within the virtual reality environment. The area servers preferably include a processor for running software responsible for administering virtual geographic areas within associated unique virtual worlds. The method preferably includes logging a user computer into the virtual reality environment through the login server, and selecting a virtual geographic area within an associated virtual reality world, and assigning one of the area servers to the user computer based on the virtual geographic area selection. The assigned area server will administer the virtual geographic area while an avatar, controlled by the user computer, remains within the virtual geographic area associated with the assigned area server. A different area server will preferably be employed when the avatar exits a particular virtual geographic area associated with a first area server, and enters a new virtual geographic area within the virtual world.
In a preferred method, the computer system is connected to a telecommunications network, and the method includes logging computers into the telecommunications network.
In a further embodiment, the computer system may include a patch server, and the method may include updating software on a user computer with the patch server.
It is preferable for the computer system of the invention to include a visual display on at least one display screen that provides statistical information about a virtual reality environment used by a plurality of users. The computer system including a telecommunications complex and a plurality of world server complexes both connected to an administration server for providing the visual display on a screen. The visual display may include a computer system status area, which provides information about the status of the computer system including the number of the users using the computer system. The visual display may also include a world status area, which provides information about the number of users using each of the world server complexes.
It is preferable for the computer system status area to further include a telecommunications status area, which includes a display of plurality of packets of data being sent and received through the telecommunications complex.
The computer system of the present invention may implement a plurality of virtual reality environments used by a plurality of users wherein the world status area further includes a virtual reality game management area for selecting one of the virtual reality worlds to be displayed in the world status area.
A related aspect of the present invention pertains to a method of creating a visual display on at least one display screen, where the visual display includes information about a multi-user virtual reality game. The at least one visual display may be reviewed and monitored remote from user computer displays by an administrator of the present virtual reality game. The method preferably comprises a utilizing a plurality of world-server complexes to create unique virtual reality worlds in which a user can interact with other users through avatars, operated by user computers connected to the world server complexes. A method also comprises utilizing an administration server connected to the plurality of world server complexes and the plurality of user computers through a telecommunications network. A visual display is provided on at least one display screen, where the visual display includes a world status area which identifies a plurality of virtual worlds and information about the number of user computers logged into the plurality of virtual worlds.
A method may comprise displaying information about the number of users who have submitted questions about the virtual reality game within the world status area. The users of the virtual reality game may be assigned at least one status level based on their achievements within the game. The method may further comprise displaying information within the world status area about the quantity of users at particular status levels logged into the plurality of virtual worlds.
The method may also comprise displaying a computer system status area within the visual display on at least one display screen, where the computer system status area identifies information about the number of users utilizing the computer system. A telecommunications status area may be displayed within the computer system status area, wherein the telecommunications status area includes information about the number of packets of data being sent and received through the telecommunications complex of the computer system.
In an embodiment where the computer system includes a plurality of routers, the method may comprise arranging a router status area as part of the computer system status area. The router status area may identify information about the overall flow of packets of data through the administration server. The router status area may also identify information about the elapsed time since the last user logged into the computer system. The router status area may also identify the average quantity of data for each user handled by routers of the telecommunications network.
In accordance with a related aspect of the present invention, a computer system is provided for implementing a virtual reality game used by a plurality of users. The computer system may comprise a telecommunications complex; a plurality of world server complexes; a plurality of user computers connected to the world server complexes through the telecommunications complex; and an administration server connected to the plurality of world server complexes and the plurality of user computers through the telecommunications complex. A visual display is created on at least one display screen for providing statistics about the computer system. The visual display preferably comprises a world status area identifying a plurality of virtual reality worlds and including information about the number of user computers logged into one or more of the plurality of virtual worlds.
The visual display may also include various information discussed above in connection with preferred methods of the present invention.
In another embodiment of the invention, a computer messaging system is provided for implementing selective retransmission of messages between communication end points of a communication channel. It is preferable for at least some of the messages to have a retransmission indicator, which provides a retransmission of the message when an initial transmission failure for the message occurs. It is also preferable for the system to include a connection object being instantiated at the communication end points. The connection object includes a timeout tracking object for calculating a timeout value related to a round-trip messaging time for the communication channel, and a bandwidth tracking object for calculating a transmission bandwidth for the communications channel. The connection object preferably also includes an outbound queue for storing the message prior to the initial transmission failure of the message over the communications channel; and an outstanding queue for storing the message to be retransmitted over the communications channel upon the initial transmission failure.
Preferably, the timeout value within the timeout tracking object includes a ready for retransmission value for determining a retransmission time for the message in the outstanding queue.
It is also preferable for the message to be sent over the communications channel when the transmission bandwidth exceeds a number of messages in the outbound queue by a threshold value.
In another embodiment, a computer messaging method and a computer readable media are provided for implementing a selective retransmission of messages through a communications channel. The messaging method may include selecting a message as a retransmittable message, and storing the selected message in an outbound queue, and sending the message over the communications channel, and storing the selected message in an outstanding queue if an acknowledgement is not received in response to the sent message, and resending the selected message.
It is preferable for the method and computer readable media discussed above to further include determining a ready for retransmission value, where the resending of the selected message occurs accord to the ready for retransmission value.
It is also preferable for the method and computer readable media discussed above to further include determining a transmission bandwidth threshold for the communications channel, where the selected message is sent occurring according to the transmission bandwidth threshold.
A preferred computer messaging system for implementing selective retransmission of messages between communication end points of a communication channel may include a group of messages to be transmitted over the communications channel. A connection object is programmed as part of a virtual reality environment for selecting messages containing significant data from the group of messages, and a retransmission object is programmed as part of the virtual reality environment for retransmitting the selected messages. The retransmission object will retransmit the selected messages only after a first unsuccessful transmission of the selected messages over the communications channel.
In another aspect of the invention, a visual display system is provided for displaying information related to a network connection. The visual display preferably includes a packet transmission bar, which provides graphical and numerical information related to the network connection. Such information may include instantaneous bandwidth information indicative of the network connection and instantaneous round trip transmission time information for the network connection. The visual display preferably also includes an outbound packet status bar within the packet transmission bar for providing a graphical outbound packet status and a numerical outbound packet loss value, the outbound packet status bar including areas representing lost outbound packets as indicated by an unacknowledged transmission over the network connection. An inbound packet status bar may also be arranged within the packet transmission bar for providing graphical inbound packet status and a numerical inbound packet loss value, the inbound packet status bar including areas representing lost inbound packets as indicated by a missing packet sequence value received over the network connection.
The areas representing the lost outbound packets of the visual display system discussed above preferably advance along the outbound packet status bar as additional packets are sent over the network connection, wherein the areas representing the lost inbound packets advance along the inbound packet status bar as additional packets are received over the network connection.
The outbound packet status bar and the inbound status bar discussed above are preferably disposed-end-to-end within the packet transmission bar, the adjacent ends representing a packet status for a most recently sent packet and a most recently received packet respectively.
It is preferable for a method associated with the visual display system discussed above to provide such system with information related to a network connection, the graphical display including a packet transmission bar having an outbound packet status bar portion and an inbound packet status bar portion. The method preferably includes displaying an outbound indicator at one end of the outbound packet status bar portion for outbound packets that are unsuccessfully sent over the network connection, and advancing the outbound indicator along the outbound packet status bar portion as additional packets are sent over the network connection. It is also preferable to display an inbound indicator at one end of the inbound packet status bar portion for inbound packets that are unsuccessfully received over the network connection, and to advance the inbound indicator along the inbound packet status bar portion as additional packets are received over the network connection.
The method and computer readable media discussed above may also provide a bandwidth value that is indicative of a bandwidth of the network connection.
The method and computer readable media discussed above may also provide a delay value indicating a round trip delay time for a packet on the network connection.
The present invention also contemplates a visual display for providing information related to a network connection including a packet transmission bar for providing a first graphical indicator related to the network connection. The packet transmission bar including an outbound packet status bar for providing a graphical indication of lost outbound packets and a numerical outbound packet loss value. The packet transmission bar also includes an inbound packet status bar for providing a graphical indication of lost inbound packets and a numerical inbound packet loss value.
It is yet another aspect of the present invention to provide a method of providing efficient transmission of data related to visual objects from a point of view of an avatar within a virtual reality environment. One preferred method may comprise providing data representative of an avatar within the virtual reality environment; determining a point of reference location of the avatar within the virtual reality environment; providing data representative of visual objects other than the avatar within the virtual reality environment; and updating the data representing visual objects other than the avatar at a frequency dependent upon the proximity of the objects to the point of reference of the avatar.
The method may also comprise providing a plurality of proximity ranges relative to the point of reference location of the avatar within the virtual reality environment. Different update frequency periods are preferably assigned to respective ones of the plurality of proximity ranges, defining the frequency at which certain updated data will be transmitted within the plurality of proximity ranges. It is also preferable for the method to comprise updating data which is representative of objects other than the avatar within the virtual reality environment at the frequency assigned to the respective plurality of proximity ranges.
A method may also comprise updating data which represents objects other than the avatar, based on relative movement of the avatar to the other objects within the virtual reality environment.
In yet another embodiment, the method may comprise assigning various priority values to the objects other than the avatar relative to each other within respective proximity ranges, whereby data representing the objects within each proximity range that is assigned a higher priority value than other data within the same proximity range will be updated more frequently than the other data.
It should be appreciated that the method in accordance with this aspect of the present invention may be related to various visual objects, rather than specifically representative of such objects. For instance, the data may relate to the state of a virtual animal, such as full of energy, injured, dying, or dead, whether the animal is moving quickly, slowly, etc.
It is yet another aspect of the invention to provide a system for implementing a bandwidth efficient communications channel between a server supporting a virtual reality environment and a computer for allowing a user to operate within the virtual reality environment. A point of reference is selected within the virtual reality environment. A plurality of proximity ranges relative to the point of reference exist within the virtual reality environment. The plurality of proximity ranges preferably include a first proximity range for an element near the point of reference within the virtual reality environment. A view frustum having a center line containing the point of reference also exists within the virtual reality environment along with a plurality of proximity ranges relative to the view frustum. A plurality of second proximity ranges exists for determining a second proximity range for the element near the point of reference within the virtual reality environment. Messages may be sent over the communications channel, the messages having data regarding a change of position for the element as perceived by the user through the view frustum from the point of reference. A plurality of update frequencies for the messages may also be sent over the communications channel, the update frequency being correlated in generally increasing relation with generally increasing proximity to the point of reference as indicated by both the first and second proximity ranges. A particular update frequency may be selected from the plurality of update frequencies for sending the messages based on the first and second proximity ranges.
The system discussed above may further include a weighting module programmed as part of the virtual reality for weighting the first and second proximity ranges in the selection of the particular update frequency. It is preferable for the data regarding a change in position to represent different types of movements for the point of reference and the view frustum, the different types of movement being represented by data having different bit lengths.
Another embodiment of the present invention provides a method of and computer readable media for determining a particular update frequency for a messaging system. The messaging system implementing a bandwidth efficient communications channel between a server supporting a virtual reality environment and a computer for allowing a user to operate within the virtual reality environment. The virtual reality environment being presented to the user on the computer with respect to a point of reference within the virtual reality environment. The method including determining a first proximity range for an element from a plurality of proximity ranges relative to the point of reference, and determining a second proximity range for the element from a plurality of proximity ranges relative to a view frustum, the view frustum having a center line containing the point of reference. A plurality of update frequencies are preferably correlated in generally increasing relation with generally increasing proximity to the point of reference as indicated by both the first and second proximity ranges. A particular update frequency is selected for messages within the messaging system from the plurality of update frequencies based on the first and second proximity ranges, the messages having data regarding a change of position for an element perceived by the user from the point of reference.
The first and second proximity ranges may be weighted in the selection of the particular update frequency. It is also preferable for data to be provided, where the bit length thereof may vary based on the representation of a type of movement for the point of reference and the view frustum.
The present invention also includes a system for implementing a bandwidth efficient communications channel between a server supporting a virtual reality environment and a computer for allowing a user to operate within the virtual reality environment. The virtual reality environment being presented to the user on the computer with respect to a point of reference within the virtual reality environment, including a circle of proximity centered on the point of reference and indicating a proximity range for an element near the point of reference within the virtual reality environment, and a field of view having a foci at the point of reference. The field of view overlapping a portion of the circle of proximity and including the element. The system contemplates messages sent over the communications channel, the messages having data regarding a change of position for the element, and a plurality of update frequencies for the messages. The update frequency being correlated with the location of the element within the circle of proximity or the field of view. The update frequencies generally increasing with generally increasing proximity to the -point of reference and the field of view.
The foregoing features of the present invention will be better appreciated with reference to the following detailed description of the preferred embodiments and accompanying drawings.